The present invention relates to a chroma keyer for creating a video special effect, and more particularly to a chroma keyer with correction for background defects such as uneven lighting level and/or color temperature, creases and wrinkles, smudges, and use of colors with variations in luminance to chrominance ratios.
A typical real setup, or chroma key set, (FIG. 2) has a foreground object or scene, such as a flower vase, placed in front of a uniform color background scene (FIG. 1), usually blue or green. The chroma key set is then shot with a video camera. The video output of the camera is modified by a chroma keying source to replace the color background scene with a new background scene (FIG. 3), such as a room with a table and other objects. The result is a visual effect that makes the foreground scene appear as though it were in front of the new background scene (FIG. 4). In film a similar technique is called "matting".
Traditional chroma keying circuits have not always been able to produce results that are free of unwanted artifacts. In particular artifacts result when the color background scene is not a perfect, uniform color and luminance. Nonuniformity in the color background scene (FIG. 1), such as uneven lighting level and/or color temperature, creases and wrinkles, smudges, and use of colors with variations in luminance to chrominance ratios, require compromises in the way that the chroma key circuit is set up. For example a common practice used in creating a chroma key effect is to place props in the chroma key set that have the same color as the color background scene. These props may represent rocks, tables, chairs, or any such object that the foreground object may sit on or move around. The props should have the same luminance to chrominance ratio in the paint applied to them so that they match the color background scene. Yet they often do not. When the props do not match, the luminance suppression in most traditional chroma key circuits becomes difficult since the luminance suppression is based on the chrominance suppression by a proportionality constant. If the luminance suppression is adjusted for the color background scene, it is not correct for the props--it is either insufficiently suppressed or oversuppressed. If the luminance suppression is adjusted for the props, then the color background scene is similarly improperly suppressed. Oversuppression causes edges of the foreground object to be degraded so that detail in the edge is lost. Undersuppression causes the new background scene in the composite image to be of incorrect luminance level. An operator has to decide which artifact is least harmful for a particular effect desired.
A similar problem applies with uneven lighting, smudges, and wrinkles in the chroma key set. For this case suppression of the color background scene works fine, but conditioning of the new background scene is compromised because a control signal from the chroma keying circuit, applied to the new background scene to cut a hole in it where the foreground object is, must be conditioned so that it does not respond to the smudges, wrinkles and lighting defects. This requires gain and clipping to be applied, and the more gain necessary, the more the edges between the new background scene and the foreground object are degraded. The edges take on a hard appearance and detail is lost.
A related problem occurs when the colored prop is placed in the chroma key set. The prop necessarily has shadows on it, or casts shadows onto other parts of the set. Even with even lighting objects in the set have the appearance of uneven lighting because of the way light reflects unevenly at different angles and from different types of surfaces. These variations also require application of clip and gain to the chroma control signal, and result in loss of edge detail.
Where the lighting does not have a uniform color temperature, most chroma key circuits have imperfect chrominance suppression because most chroma key circuits use a pure color in the suppression process. The residue of imperfect chrominance suppression adds to the new background scene, causing areas in the composite image to be polluted with hue errors (FIG. 4).
So chroma key sets with defects in the color background scene as described above cause undesirable artifacts in the resulting chroma key effect. One would like to make the color background scene ideal, i.e., completely uniform from the point of view of the camera, but in practice this is not possible by making adjustments to the set or its lighting, especially when props are used.
U.S. Pat. No. 5,032,901 issued Jul. 16, 1991 to Petro Vlahos entitled "Backing Color and Luminance Nonuniformity Compensation for Linear Image Compositing"; incorporated herein by reference describes two methods for correcting chroma key operation based on a reference image of the color background scene without the foreground object in place. The circuit shown in FIG. 13A U.S. Pat. No. 5,032,901 has a block 121 that generates the chroma key signal used to shape the new background scene, using a hue selector and a reference image. The hue selector is placed before an embedded storage device that normalizes the output from the hue selector which is used only for shaping the new background scene. Circuits 112, 115 of FIG. 13A show suppression circuits including correction for defects in the color background scene. The approach disclosed is similar to a traditional chroma key circuit, i.e., it shapes a uniform matte signal and a constant luminance to chrominance ratio with the output of a traditional hue selector, which responds to the defects in the color background scene, and subtracts the result from the color background scene. When the imperfect color background scene without the foreground object is suppressed, the result of the suppression is an error due to the defects. The error is stored in a memory and shaped by the output of the hue selector so that it does not respond to the foreground object. When the color background scene with the foreground object is processed, the shaped error output from the memory is used to remove the residual error in the color background scene. FIG. 10 of U.S. Pat. No. 5,032,901 discloses a circuit for normalizing the color of the color background scene for input to a traditional chroma keyer.